Skip to content

[SPARK-55992][SQL] Fix GroupPartitions textual representation#54801

Closed
peter-toth wants to merge 1 commit intoapache:masterfrom
peter-toth:SPARK-55992-fix-grouppartitions-representation
Closed

[SPARK-55992][SQL] Fix GroupPartitions textual representation#54801
peter-toth wants to merge 1 commit intoapache:masterfrom
peter-toth:SPARK-55992-fix-grouppartitions-representation

Conversation

@peter-toth
Copy link
Contributor

What changes were proposed in this pull request?

This PR improves GroupPartitionsExec representation in plans.

Before this PR:

GroupPartitions ArraySeq(0), List((org.apache.spark.sql.catalyst.util.InternalRowComparableWrapper@379fae8f,1), (org.apache.spark.sql.catalyst.util.InternalRowComparableWrapper@dea578e3,1)), ArraySeq(Some(BucketReducer(2))), false

After this PR:

GroupPartitions JoinKeyPositions: [0] ExpectedPartitionKeys: 2 Reducers: 1 DistributePartitions: false

Why are the changes needed?

expectedPartitionKeys might be huge and as it is based on InternalRow it doesn't make sense to display.

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Added new UT.

Was this patch authored or co-authored using generative AI tooling?

No.

@peter-toth
Copy link
Contributor Author

cc @dongjoon-hyun, @szehon-ho

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @peter-toth .

Sorry for the delay. I was sick during the weekend.

@peter-toth
Copy link
Contributor Author

+1, LGTM. Thank you, @peter-toth .

Sorry for the delay. I was sick during the weekend.

Thank you @dongjoon-hyun , no problem. Hope you feeling better now.

@szehon-ho
Copy link
Member

szehon-ho commented Mar 17, 2026

hi @peter-toth @dongjoon-hyun , i took a look at the pr, and had some small comment. As its merged, I just made a pr here: #54842 , can you take a look?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants